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Abstract 

This study is a secondary data analysis of the USEIT data to inform school 
administrators and policymakers about the factors affecting instructional 
technology use in elementary classrooms. Researchers developed a predictive 
multilevel SEM model for teacher-directed student use of technology (TDS). 
The model depicts relationships between factors across district, school, and 
classroom levels of analysis. The strongest predictors of TDS were (a) teach¬ 
ers’ experience with technology, belief that technology is beneficial to meet 
instructional goals, and perceived pressure to use technology; (b) principals’ 
use of technology; and (c) technology standards, teacher and student ac¬ 
countability to standards, and principals’ technology discretion. (Keywords: 
instructional technology, educational technology, computers, diffusion of in¬ 
structional technology, and technology leadership) 


S ince the early 1990s, schools, districts, and the federal government have 
invested heavily in instructional technology (IT). Teacher and student 
access to technology in schools has improved dramatically. Today, all 
public schools are connected to the Internet, with 97% connected via high¬ 
speed connection. The student-to-computer ratio dropped from 4.4 in 2003 
to 3.8 in 2005 (Wells & Lewis, 2006), and hundreds of schools and districts 
are experimenting with or have put in place one-to-one laptop programs that 
provide each student with their own laptop. Yet, evidence suggests that invest¬ 
ments in IT may not have translated into widespread use in schools (Cuban, 
2001; Russell, Bebell, O’Dwyer, & Miranda, 2003a; Russell, Bebell, O’Dwyer, 

& O’Connor, 2003b; Wells & Lewis, 2006). Diffusion theories provide insight 
into factors related to schoolteachers’ disappointing adoption rates of instruc¬ 
tional technologies (IT) (Miranda, 2007). Rogers’ (1995) theory of diffusion 
and Surry and Farquhar’s (1997) application of diffusion theory to IT sug¬ 
gest that the limited use of instructional technologies by teachers and their 

Volume 43 Number 4 | Journal of Research on Technology in Education | 301 
Copyright ©2011, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, iste.org. All rights reserved. 




Miranda & Russell 


students may stem from the implementation of programs that address only a 
portion of the many aspects of diffusion. 

The literature on diffusion indicates that to promote IT use, implementa¬ 
tion plans should incorporate both micro-level and macro-level aspects of 
diffusion (Miranda, 2007). Micro-level theories that focus on adopter charac¬ 
teristics and processes follow an instrumentalist philosophy, which argues that 
the needs of individuals drive social change (Surry & Farquhar, 1997). This 
suggests that teachers’ and students’ needs drive the development of innova¬ 
tions and influence the extent to which teachers adopt instructional technolo¬ 
gies. In contrast, Surry and Farquhar (1997) argue, macro-level theories focus 
on organizational characteristics and follow a deterministic philosophy, which 
contends that developers supply technological innovations and drive social 
change. Macro-level theories suggest that educational experts, industry, state 
education organizations, and district leaders define needs, develop solutions, 
and enforce technology use through policies, accountability, and regulations. 

According to Surry and Farquhar (1997), top-down, macro-level ap¬ 
proaches to IT adoption are rooted in deterministic philosophy, which sees 
technology as an autonomous driving force for social change that is outside 
human control. An extreme example of a technology program that employs 
only a macro-level approach while ignoring micro-level aspects is a school 
district that decides to purchase and implement IT—e.g., a classroom grad¬ 
ing system, virtual science experiments, a diagnostic reading program— 
without first assessing classroom needs or involving teachers in the selection 
process. Macro-level approaches are likely to fail because teachers may not 
see the innovation’s benefits and resist its adoption. 

In contrast, Surry and Farquhar (1997) contend that micro-level ap¬ 
proaches are rooted on instrumentalist philosophy, which believes humans 
are the driving force for social change and technology is a tool of change 
used by humans. Whereas deterministic philosophers view technology 
adoption as a revolutionary process that happens in leaps, instrumentalists 
view it as an evolutionary process that is slow, gradual, and initiated by the 
needs of adopters. An example of a program that employs only a micro-level 
approach is a school district that relies on teachers to identify needs, apply 
for external grants, and develop local partnerships with technology leaders 
to implement new technologies. This tends to result in the implementation 
of a diverse range of technologies across classrooms implemented without 
coordinated strategies for support. This type of approach rarely translates 
into widespread technology-infused instructional practices. 

As the examples above illustrate, employing either a micro-level approach 
or a macro-level approach may limit IT adoption (Surry & Farquhar, 1997). 
Thus, given the importance of micro and macro aspects of diffusion to the 
adoption of an innovation, efforts to identify factors that influence instruc¬ 
tional use of technology may benefit from a framework that integrates 
principles of both micro and macro theories of diffusion. 
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Research on educational technology identifies several factors that may 
influence the use of instructional technologies. Some of these factors reside 
at the organizational (e.g., school and/or district) or macro level, whereas 
others are specific to teachers and students and are classified as micro-level 
factors (Miranda, 2007). Educational technology literature points to district- 
level factors, such as resources, funding, leadership, vision, and technology 
planning, as important drivers of educational technology use. Research 
indicates that as resources and funding increase, so does the frequency of 
technology use (Anderson & Ronnkvist, 1999; Becker, Ravitz, & Wong, 

1999; Fisher, Dwyer, & Yocam, 1996; Lemke, Quinn, Zucker, & Cahill, 

1998). Evidence also suggests that leaders who have a well-defined technolo¬ 
gy vision and who develop clear technology plans tend to encourage the use 
of technology, resulting in increased frequency to instructional technology 
in their districts (Anderson & Dexter, 2005; Dawson & Rakes, 2003; Fisher 
et al„ 1996; Lemke et al. 1998). 

The literature features principal leadership, availability of professional 
development for principals, and perceived pressure to use technology among 
school-level factors related to instructional technology use. Evidence sug¬ 
gests that at the school-level leadership appears to be an important con¬ 
tributor to multiple uses of technology by teachers and students (Anderson 
& Dexter, 2005; Dawson & Rakes, 2003; Russell et al„ 2003a). In light of 
school-level leaderships importance in driving teachers’ and students’ use 
of technology in schools, principals’ technology training, particularly when 
focused on integration (Anderson & Dexter, 2005; Dawson & Rakes, 2003), 
also appears to be an essential condition to increase the use of educational 
technology. Russell et al. (2003b) also identified perceived pressure to use 
technology as a factor associated with technology use for multiple purposes. 
Russell et al. (2003b) reported perceived pressure to use technology at the 
school level as one of the predictors of teacher’s use of technology for deliv¬ 
ering instruction and for teacher-directed student use of technology during 
class time. 

Within classrooms, micro-level factors cited in the literature include 
teacher-related characteristics as well as access to technology. Several authors 
report a teacher’s educational philosophy as a possible contributor to educa¬ 
tional technology use. This line of work reports that teachers who hold more 
constructivist teaching beliefs are more apt to use technology for educational 
purposes (Becker, 2000; Becker et al„ 1999; Ertmer, Gopalakrishnan, & Ross, 
2001; Fisher et al., 1996) than are teachers who hold less constructivist views. 
Likewise, the literature identifies teachers’ beliefs about technology as a predic¬ 
tor of educational technology use. Seemingly, teachers who believe technol¬ 
ogy is valuable and beneficial for teaching and learning are more likely to use 
technology more frequently than are teachers who do not hold such beliefs 
(Ertmer, 2005; Ertmer, Ottenbreit-Leftwich, 2010; Russell et al., 2003b). The 
literature also refers to teachers’ access to and experience and comfort level 
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with technology as possible predictors of technology use. Essentially, teach¬ 
ers who have more access to technology and who have more experience with 
technology appear to be more comfortable with technology and, therefore, 
use technology more frequently than teachers who have less access and less 
experience (Becker et al., 1999; Miranda, 2007; Russell, O’Dwyer, Bebell, & 
Tao, 2007). Finally, at the classroom level, evidence indicates that there may 
be a link between teachers’ background characteristics, such as number of 
years teaching and educational background factors, and frequency of tech¬ 
nology use. Evidently, teachers who have strong academic backgrounds (i.e., 
college selectivity, level of degree earned, and advanced courses completed) 
appear to be more frequent technology users of technology than are teachers 
with weaker academic backgrounds (Becker, 2000; Becker et al. 1999; Becker 
& Riel, 2000; Guha, 2000; Mathews, & Guarino, 2001; Russell et al., 2003b). 

Despite the breadth of research on IT, there is little empirical evidence 
of the magnitude of the influence factors listed above have on the use of IT. 
Research has focused on these factors in isolation, ignoring the interactions 
among the factors. Additionally, with the exception of O’Dwyer, Russell, 

& Bebell (2005), prior studies do not take into account that factors reside 
at different hierarchical levels within school systems. Findings from past 
research may misrepresent the importance of specific factors in shaping 
instructional use of technology and are minimally useful in helping districts 
shape technology initiatives. 

The study presented here employs multilevel modeling techniques to 
examine the direct and indirect effects that a variety of macro- and micro- 
level factors have on instructional use of technology in the classroom and 
addresses the following questions: 

• Which district-level factors have the largest effect on teachers’ instruc¬ 
tional use of technology? 

• Which school level factors have the largest effect on teachers’ instruc¬ 
tional use of technology? 

• Which classroom-level factors have the largest effect on teachers’ instruc¬ 
tional use of technology? 

• How do factors interact within and across levels of a school level to affect 
teachers’ instructional use of technology? 

The authors define instructional technology as digital or computer tech¬ 
nologies (e.g., computers, CDs, DVDs, LCD projectors, interactive media, 
Internet) used for teaching and learning in the classroom. The operational 
definition of IT use in this study followed Bebell, Russell, and O’Dwyer’s 
(2004) framework for teacher technology. Researchers used the Teacher-Di¬ 
rected Students’ Use of Technology (TDS) scale developed for the Use, Sup¬ 
port, and Effect of Instructional Technology (USEIT) study as the outcome 
measure (refer to Method section). 
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Method 

The study discussed in this article consisted of secondary data analyses of 
USEIT data (Russell et al, 2003a). The USEIT study originated from a field 
initiated study requested by 16 Greater Boston school districts and ultimate¬ 
ly included 21 Massachusetts school districts. The USEIT sample consisted 
of a convenience sample of districts whose leaders wanted to assess the use 
of technology as well as conditions that supported or hindered the use of 
technology in their respective schools. The USEIT research team mailed 
surveys to district administrators, principals, and teachers in the districts 
interested in the Field Initiated study. The survey response rates were 96% 
of district administrators, 80.5% for principals, and 69% for teachers. The 
USEIT study examined technology programs in 3 small urban, 5 rural, and 
13 suburban districts located in Massachusetts. Table 1 presents a compari¬ 
son of Massachusetts’ districts by locale type districts in the USEIT sample 
and shows that 14% of the districts in the USEIT sample were classified as 
urban, 62% were suburban, and 24% were rural. 

Table 2 (p. 306) presents a comparison of sample demographics, including 
type of school, percent of white students in the district, and percent of stu¬ 
dents in the district receiving free or reduced lunch to state demographics. 

Tables 1 and 2 show that the number of suburban districts is over-repre¬ 
sented in the USEIT sample, whereas the number of urban districts is under¬ 
represented; and the number of white students is over-represented, whereas 
the number of students on free and reduced lunch is under-represented when 
compared to Massachusetts’ districts. Note that USEIT researchers did not aim 
to generalize findings from this study but intended to investigate which dis¬ 
trict- and school-level supports translated to regular use of IT in the interested 
21 districts. The USEIT district sample was a purposive sample designed to 
respond to the needs of a group of school districts in Massachusetts. Gener¬ 
alization of findings from the study discussed here is limited to schools with 
characteristics similar to those in the USEIT sample. Despite these limitations, 
the USEIT study is one of the most comprehensive educational technol¬ 
ogy studies conducted to date that includes multiple data sources collected 
at different hierarchical levels (i.e., district, school, classroom, and student). 
The USEIT study included surveys of district-level personnel, principals, 
and teachers, as well as site visits to each of the districts. This study employs 


Table 1. Percentages of School Districts by Locale Type in Massachusetts 


District Type 

Massachusetts 

USEIT Sample 

Urban 

29.5% 

14% 

Suburban 

51% 

62% 

Rural 

19.5% 

24% 
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Table 2. Sample Demographics and Massachusetts Demographics Overal (Russel et al., 2003a) 


District 

Elementary 

Middle 

High 

Race 
(% White) 

SES 

(% Free Lunch) 

A 

5 

0 

0 

87 

3 

B 

0 

1 

1 

88 

1 

C 

7 

1 

1 

87 

10 

D 

3 

1 

1 

96 

7 

E 

2 

1 

1 

89 

4 

F 

4 

1 

1 

86 

7 

G 

1 

0 

0 

89 

3 

H 

1 

1 

0 

94 

0 

1 

3 

1 

0 

90 

4 

J 

0 

0 

1 

86 

4 

K 

7 

0 

1 

74 

37 

L 

6 

1 

1 

96 

6 

M 

2 

1 

1 

97 

8 

N 

0 

0 

1 

94 

9 

O 

16 

4 

2 

82 

5 

P 

4 

1 

1 

95 

28 

Q 

6 

1 

1 

87 

9 

R 

6 

1 

1 

88 

5 

S 

2 

1 

1 

99 

2 

T 

4 

1 

1 

96 

4 

U 

9 

2 

1 

65 

26 

V 

3 

1 

1 

92 

19 

W 

3 

1 

1 

81 

3 

X 

5 

1 

1 

93 

3 



Totals 




USEIT 

104 

23 

22 

89 

9 

Massachusetts 

1270 

282 

318 

76 

25 


data from each of these sources. The section below describes the sample, data 
sources, and analytic procedures. 

Sample 

District-level data for this study are based on survey responses from 104 
district-level personnel, which included 18 superintendents, 1 assistant 
superintendent, 32 curriculum development directors, 15 head technicians, 
17 professional development directors, and 21 directors of technology. 
Researchers collected school-level data from 81 elementary principals and 
classroom-level data from 1,040 teachers across the 81 elementary schools 
for which there was corresponding principal data. Of the 1,040 teachers 
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surveyed, 14% taught kindergarten, 16% taught first grade, 18% taught 
second grade, 20% taught third grade, 19% taught fourth grade, and 13% 
taught fifth grade. Most teachers indicated they were veteran teachers with 
approximately 58% reporting that they had been teaching for more than 10 
years. Approximately 84% of the teachers reported having Internet access in 
their classrooms, and 39% reported having access to three or more desktop 
computers in their classrooms. Approximately 14% of the teachers reported 
that they did not have access to desktop computers in their classrooms, and 
of this percentage, 35% had access to computers either in a lab/media center 
or in the library. 

Data Sources 

The authors used the following USEIT data sources for this study: (a) the 
district survey, (b) the principal survey, (c) the teacher survey, and (d) site 
visit ratings. The district, principal, and teacher surveys included a large 
number of items that focused on a variety of issues related to IT. Some of 
these issues included the ways and frequency with which instructional 
technologies were used, availability and participation in professional de¬ 
velopment related to IT, beliefs about teaching, learning and IT, obstacles 
to technology use, pressure to use IT, resources and support for IT, leader¬ 
ship for IT, community support for IT, and demographics. A large number 
of measures were created using items from each of the survey instruments 
from the USEiT study (see Russell et al., 2003a). Analyses presented here 
also used rating scores for several factors obtained from site visits to each of 
the districts (see Russell et al., 2003a). 

Table 3 (p. 308) presents a summary of USEIT scales that were used in 
analyses and their respective data sources. Table 3 shows that this study 
included measures for eight district-level factors, three school-level factors, 
and eight classroom-level factors. Researchers used a scale, Teacher-Directed 
Student Use of Technology in the Classroom (TDS), which measures the 
extent to which students use technology in the classroom to complete work 
assigned by teachers (see Table 3 for reliability and factor scores) as the out¬ 
come measure for these analyses. Technology use in this instance refers to 
a variety of student uses that include individual work, group work, research 
work, writing, and so forth. This scale included the following items: (a) How 
often do students work individually using computers? (b) How often do 
students work in groups using computers? (c) How often do students use a 
computer or portable writing device for writing? (d) How often do students 
research/use the Internet or CD-ROM? (e) How often do students use a 
computer to solve problems? 

Analytic Procedures 

Researchers used structural equation modeling (SEM) and multilevel SEM 
to develop a single multilevel predictive model of TDS and three single-level 
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Table 3. Summary of Scales and Data Sources 


Type of Variable 

USEIT Scale 

Reliability * 

Data Source 

Outcome variable 

Teacher-directed student use of technology in the 
classroom (TDS) 

.84 

Teacher survey 

District-level predictors 

Funding 

.60 

Site visit data 


Resources 

.41 

Site visit data 


Evaluations consider technology 

.61 

District survey 

School-level predictors 

Principal’s beliefs 

.47 b 

Principal survey 


Principal’s use of technology 

.75 

Principal survey 


Principal’s access to professional development 

.60 

Principal survey 

Classroom-level predictors 

Beliefs about the breadth of technology’s benefits 

0.95 

Teacher survey 


Teacher pedagogical beliefs 

0.63 

Teacher survey 


Teacher's confidence with technology 

0.85 

Teacher survey 


Pressure to use technology 

0.78 

Teacher survey 


Obstacles to integrating technology into lesson plans 

0.62 

Teacher survey 


Perceived need for professional development for the 
integration of technology 

0.73 

Teacher survey 


Importance of computers for teaching scale 

0.79 

Teacher survey 


Teachers’ experience with technology 

0.71 

Teacher survey 


a Coefficients in normal font are Cronbach alphas and coefficients in italics are Cohen’s kappas, 
b Scale excluded from analyses due to poor reliability. 


models of TDS: a classroom model, a school model, and a district model. 
Structural equation modeling is a family of “a priori” statistical techniques 
used to test hypotheses about the relationships between variables (Hoyle, 
1995; Kline, 1998). In SEM, path diagrams, which are graphical representa¬ 
tions of the hypothetical model, represent hypotheses to test relationships 
among variables (Hoyle, 1995). In path diagrams, analysts use rectangles to 
depict observed variables, ellipses to depict latent variables and measure¬ 
ment error, and arrows to indicate association between variables. Similar 
to regression, error terms represent the amount of variance in outcome 
variables that is not explained by the model and are graphically represented 
in the path diagram by circles linked to each endogenous variable (i.e., vari¬ 
ables predicted by the model). Analysts use straight arrows to indicate the 
association between a predictor and an outcome variable and curved arrows 
to indicate correlations (Hoyle, 1995). Paths represent direct effects between 
predictor and outcome variables, whereas path coefficients are statistical 
estimates of relationships between variables, which the reader may interpret 
as regression coefficients (Kline, 1998). 

As Hox (2002) points out, educational researchers model complex 
relationships between latent and/or observed variables using multilevel 
SEM models. In addition to facilitating the examination of direct, indirect, 
and reciprocal effects in a single analysis, as in HLM multilevel SEM has 
the added benefit of examining relationships between outcome variables at 
lower levels of analysis (i.e., at the individual level) and explanatory variables 
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at higher hierarchical levels (i.e., at the group level). However, unlike HLM, 
SEM allows variables to be modeled as outcomes and predictors in separate 
but interrelated equations (Hox, 2002; Kline, 1998). 

Researchers used a three-step process to generate a multilevel structural 
equation model (SEM) that estimated the magnitude of direct and indirect 
effects of district, school, and classroom-level factors on teacher-directed 
student use of technology in the classroom. These steps included correla¬ 
tion analyses, generation of single-level SEM models, and development of a 
multilevel model. 

Correlation analyses. The first step in these analyses was running correla¬ 
tion analyses to investigate the strength of the relationships between TDS 
and other USEIT scales and to identify relationships among other predictor 
variables. To maximize the initial pool of possible predictors, researchers 
used a correlation of at least ±0.2 with TDS as the criterion to retain variables 
as predictor variables (Fraenkel & Wallen, 2003). To evaluate the relationship 
between TDS and school- and district-level USEIT variables, researchers ag¬ 
gregated the value of TDS to the appropriate hierarchical level. At the school 
level, researchers assigned each school the mean TDS score for teachers 
within that school. At the district level, researchers assigned each district the 
mean TDS score for teachers within that district. Researchers used the same 
aggregation procedure for classroom-level predictor variables of TDS that 
they used at higher-levels of analysis. Researchers aggregated teachers’ expe¬ 
rience with technology at the school level by calculating the mean scale score 
for teachers within the school. Schools’ mean experience with technology 
scores were then correlated with the school mean TDS scores and with other 
school-level variables, such as the principal’s use of technology. 

Generation Of SEM path models. Analyses described here used an iterative 
process to develop single level SEM models for TDS. Researchers developed 
separate single-level models for the classroom, school, and district-levels. 
The development of each single-level path model began by specifying 
a hypothetical path model (e.g., hypothesized model for TDS based on 
classroom-level factors) based on prior research findings and on correlation 
analyses; it was then estimated using AMOS. Since analyses presented here 
aimed to develop a multilevel path mode and to estimate effects at differ¬ 
ent levels within a school system, SEM analyses used covariance matrices. 
AMOS provides several fit indices that provide estimates for the extent to 
which the hypothetical model deviates from the theoretical model. Re¬ 
searchers used overall fit statistics, absolute fit indices, and incremental fit 
indices to evaluate model fit. Overall fit indices used were the chi-square 
statistic (y 2 ) and the chi-square degrees of freedom ratio (y 2 /DF). Absolute 
indices used were the goodness of fit index (GFI) and the adjusted goodness 
of fit index (AGFI). Incremental fit indices used were the normed fit index 
(NFI) and the comparative fit index (CFI). Table 4 presents the evaluation 
criteria used. 
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Table 4. Criteria to Accept the Hypothesized Model 


Index 

Evaluation Criterion 

X 2 

Small x 2 »ith probability >0.05 

X 2 /DF 

Ratios of 3 or less 

GFI 

>0.90 

AGFI 

>0.90 

NFI 

>0.95 

CFI 

>0.95 


When researchers accepted a model based on fit criteria presented in 
Table 4, they terminated estimation and selected that model as the final 
model for TDS at that particular level of analysis. Analyses proceeded by 
developing a model at the next hierarchical level. When fit indices suggested 
that improvements could be achieved, researchers used modification indices 
provided by AMOS to guide model respecification. According to Byrne 
(2001), modification indices (MI) “reflect the extent to which the hypoth¬ 
esized model is appropriately described” (p. 90) and to what extent the 
estimate model may be improved if specific parameters are specified. AMOS 
provides an MI for each fixed parameter, which represents the expected y 2 
drop if the parameter were estimated. The researchers repeated this process 
of modifying a model and examining fit indices until they obtained an ad¬ 
equately fitting single-level model at each level. 

Multilevel model development. The researchers used relationships found 
in the single-level SEM models (i.e., classroom, school, and district) to inform 
the development of a multilevel model. Analyses reported here represent a 
multilevel model in a single diagram that contains paths between predictor 
variables at each level of analyses and the outcome variable. In the multilevel 
model, each path contains three path coefficients, which one may interpret as 
regression coefficients. To create datasets for each level of analysis, researchers 
aggregated variables from lower levels, assigned the mean score to the higher 
level, and assigned variables from higher levels to lower levels. As an example, 
for school-level analysis, researchers aggregated classroom-level variables (e.g., 
teacher beliefs about technology) to the school level by calculating the mean 
scale score for teachers within a given school and then assigning that mean to 
the school level. Conversely, at the classroom level, researchers assigned values 
for school-level variables (e.g., principal’s use of technology) to each teacher, 
which remained constant across all teachers within a school. Table 5 presents a 
summary of variables used in analyses. 

Analyses proceeded with model building. Multilevel model build¬ 
ing employed a modified version of Hox’s (2002) group approach. Hox 
(2002) recommends beginning multilevel model development at the 
lowest level of analysis and then proceeding with the introduction of 
variables at higher levels. Following this procedure, one arrives at the 
optimal model if adding a variable adds little or no improvement be- 

310 I Journal of Research on Technology in Education | Volume 43 Number 4 
Copyright ©2011, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 



Teacher-Directed Student Use of Technology 


Table 5. Summary of Scales in Multilevel Analyses by Dataset 


Values Used for the Scales by Dataset 


Scales Used in Analyses 

Classroom 

School 

District 

TDS 

Individual a 

Aggregated b 

Aggregated b 

Positive technology beliefs 

Individual a 

Aggregated 6 

Aggregated b 

Technology experience 

Individual a 

Aggregated “ 

Aggregated b 

Pressure 

Individual a 

Aggregated b 

Aggregated c 

Integration obstacles 

Individual a 

Aggregated b 

Aggregated c 

Technology standards 

Disaggregated d 

Disaggregated' 

District's value 1 

Principal’s discretion 

Disaggregated d 

Disaggregated ■ 

District's value 1 


a Individual teacher’s value on the scale was used 
b Mean value on the scale for teachers within a school was used 
c Mean value on the scale was used 

d District’s value on the scale used as a constant across teachers within a district 
e District’s value on the scale used as a constant across principals within a district 
f District’s value on the scale was used 


tween two consecutive models. One evaluates model improvement by 
examining the chi-square drop between two consecutive models and the 
probability associated with chi-square, and by comparing the models’ 
fit indices. The authors intended to begin multilevel model develop¬ 
ment at the classroom stage, but it was evident during the single-level 
model development stage that multicollinearity issues would arise if we 
promoted all classroom-level variables to the school level. Consequently, 
our multilevel model development began at the school level. The authors 
used the final school-level model obtained during single-level analyses 
(see Figure A1 in the Appendix on p. 322) as the base for the multilevel 
model. The initial model contained separate path coefficients for each 
level of analysis. 

After estimation, using criteria listed in Table 4, researchers evaluated fit 
indices to assess the model’s adequacy. When the authors judged a model as 
adequate, they introduced the best predictor of TDS at the next hierarchical 
level. If the model did not fit the data adequately, modification indices and 
theory guided the addition and removal of paths or variables. The research¬ 
ers evaluated modification indices first for the classroom level of analysis, 
followed by the school level, and finally by the district level. As suggested by 
Rowe and Hill (1998) and by Hox (2002), multilevel modeling stopped when 
adding a new variable did not improve model fit (i.e., if the chi-square of the 
model with the new predictor variable was larger than the model without 
the variable). 


Results 

The authors designed the study presented here to examine direct and indi¬ 
rect effects of a variety of factors residing at different organizational levels 
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Figure 1. Final multilevel model for TDS. 


within a school system on instructional use of technology in the classroom. 
To this end, the authors developed a multilevel SEM model following proce¬ 
dures described above. 

Figure 1 presents the final multilevel model for TDS. It shows the rela¬ 
tionships between eight variables: 

• TDS 

• Teacher’s experience with technology (technology experience) 

• Beliefs about the breadth of technology’s benefits (positive technology 
beliefs) 

• Obstacles to integrating technology into lessons (integration obstacles) 

• Pressure relating to the use of technology (pressure) 

• Principal’s technology use (principal’s use) 

• The availability of and accountability to technology standards (technology 
standards) 

• Principals’ technology discretion scale (principal’s discretion) 

In practical terms, this model depicts factors that are associated with 
IT use in the classroom and that may have positive or negative effects (i.e., 
increase or decrease) on TDS. The outcome variable, TDS, resides at the 
classroom level and represents the frequency with which teachers direct 
their students to use technology in the classroom during instruction. Two 
predictors reside at the district level, one resides at the school level, and 
four reside at the classroom level. District-level variables include princi¬ 
pal’s technology discretion (principal’s discretion) and availability/enforce¬ 
ment of technology standards (technology standards). The school-level 
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Table 6. Comparison of Fit Indices for the Final Multilevel Model with District-Level Predictors (Technology Standards and 
Principal's Discretion) 


Model 

X 2 

DF 

P 

X 2 /DF 

GFI 

AGFI 

NFI 

CFI 

Multilevel model with technology standards and 
principal’s discretion 

3.50 

9.00 

.94 

0.39 

1.00 

.99 

1.00 

1.00 

Saturated 

0.00 

0.00 



1.00 


1.00 

1.00 

Independence 

812.28 

84.00 

.00 

9.67 

.85 

.81 

.00 

.00 


Note: See Table 5 for evaluation criteria. 


variable is principal’s use of technology (principal’s use). Classroom-level 
variables included teachers’ beliefs about the variety of benefits of IT (posi¬ 
tive technology beliefs), teacher’s experience with technology (technology 
experience), perceived pressure related to technology use (pressure), and 
obstacles to integrating technology into lesson plans (integration obsta¬ 
cles). The model also depicts direct effects between the following variables 
and TDS: pressure, positive technology beliefs, integration obstacles, prin¬ 
cipal’s discretion, and technology experience. Note that a direct effect is a 
relationship between a predictor and an outcome that does not involve the 
intervention of a moderator or mediating variable. The model also shows 
direct effects between several sets of variables (e.g., integration obstacles 
and pressure, positive beliefs and technology experience, and technology 
experience and positive beliefs). 

Model Evaluation 

The final model included two district-level variables: principal’s discretion 
and technology standards. Note that, although the inclusion of principal’s dis¬ 
cretion did not improve model fit, it did increase the amount of variance ex¬ 
plained and was consistent with the theoretical framework, so it was included 
in the final model. Table 6 presents fit indices for the final multilevel model. 

Assessment of fit indicated that the final multilevel model fit the theo¬ 
retical model well. The \ 2 was relatively small and nonsignificant (y 2 =3.50, 
p=0.94). y 2 /DF was 0.40 (below the recommended threshold of three), and 
all other fit indices indicated that the hypothesized model closely approxi¬ 
mated the theoretical model. The obtained GFI and AGFI were greater than 
0.9; the NFI and CFI were very close to one, indicating a very good model 
fit. When researchers added the predictor with the next highest correlation 
with TDS (external funding) to the model, its inclusion did not explain addi¬ 
tional variance, nor did it improve model fit (x 2 Finai model =3-50; x 2 E xternai funding 
m0 d e i = 5.21; see Figure A2 and Table A2 in the Appendix, p. 323). Therefore, 
the authors stopped model building and concluded that the model presented 
in Figure 1 was the best fitting model for TDS. 

Final Multilevel Model 

Figure 2 shows the final multilevel model for TDS with standardized path co¬ 
efficients, which may be interpreted as regression weights. The scales include: 
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o.oi 

0.06 

0.33 



• TDS 

• Teacher’s experience with technology (technology experience) 

• Beliefs about the breadth of technology’s benefits (positive technology 
beliefs) 

• Obstacles to integrating technology into lessons (integration obstacles) 

• Pressure relating to the use of technology (pressure) 

• Principal’s technology use (principal’s use) 

• The availability of technology and accountability to technology standards 
(technology standards) 

• Principals’ technology discretion scale (principal’s discretion) 

As shown in Figure 2, the multilevel model shows relationships between 
and among scales at three levels of analysis (classroom, school, and district). 
Each path represents relationships between variables and has three coef¬ 
ficients, which may be interpreted as regression coefficients, representing 
standardized direct effect (b) between the predictor and outcome at each 
level of analysis. Top coefficients represent classroom-level effects, middle 
coefficients represent school-level effects, and bottom coefficients represent 
district-level effects. The model does not show path coefficients less than 
0.10 as their effects are negligible (Kline, 1998). Finally, the figure shows 
squared multiple correlations for each of the scales in italics, which may be 
interpreted as the amount of variance in outcome variables explained by pre¬ 
dictors. Table 7 presents standardized total effects. 

Standardized total effects of predictor variables on TDS at the classroom 
level ranged from -0.02 to 0.25. Predictor variables that had the strongest 
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Table 7. Standardized Total Effects across Levels of Analysis b 


Scale 

Level' 

Principal’s 

Discretion 

Technology 

Standards 

Integration 

Obstacles 

Pressure 

Principal’s 

Use 

Technology 

Experience 

Positive 

Beliefs 

Technology 

Standards 

C 

0.18 

(0.02) 








S 

0.11 

(0.05) 








D 

0.27 

(0.06) 







Integration 

Obstacles' 

S 

-0.11 

(0.04) 

-0.33 

(0.07) 







D 

-0.30 

(0.02) 

-0.50 

(0.03) 






Pressure' 

S 

0.16 

(0.05) 

0.17 

(0.10) 







D 

0.23 

(0.03) 

0.44 

(0.05) 

-0.37(0.18) 





Principal’s 

Use' 

S 

-0.21 

(0.10) 








D 

0.36 

(0.05) 

0.11 

(0.08) 






Positive 

Beliefs 

C 



-0.16(0.03) 

0.17(0.03) 

-0.13 

(0.03) 

0.10(0.03) 



S 



-0.30(0.11) 

0.33 

(0.0.8) 

0.20 

(0.04) 

-0.14(0.86) 



D 


0.18 

(0.05) 

-0.52(0.17) 

0.54 (0.09) 

-0.26 

(0.06) 



TDS 

C 



-0.20 (0.03) 

0.20 (0.03) 


0.20 (0.026) 

0.25 

(0.026) 


S 



-0.34 (0.08) 

0.31 (0.06) 


0.40 (0.06) 

0.50 (0.06) 


D 

0.13 

(0.03) 

0.27 

(0.06) 

-0.50 (0.20) 

0.14(0.12) 

0.12 

(0.06) 

0.40(0.12) 

0.30(0.12) 


a C denotes classroom-level, S denotes school-level, and D denotes district level, 
b Numbers inside parentheses are standard errors 
c No Classroom effects detected 


standardized total effect on TDS included positive technology beliefs ((3 = 
0.25), pressure ((3 = 0.20), technology experience ((3 = 0.20), and integration 
obstacles ((3 = -0.20). Note that integration obstacles’ effect was negative, 
whereas other variables had positive effects. Results suggest that, at the 
classroom level, predictor variables may have small to moderate effects 
on TDS. Integration obstacles appear to have moderate negative effects 
on TDS and on two important predictors of TDS, technology experience 
and positive technology beliefs. Results suggest that teacher factors such 
as beliefs about IT’s benefits and experience with technology may increase 
the extent to which teachers direct their students to use technology, and 
that teachers whose principals emphasize IT use may increase the extent 
to which they direct students to use technology. Conversely, teachers who 
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experience obstacles with IT integration may be less likely to direct their 
students to use technology than teachers who do not experience such 
obstacles. 

At the school level, standardized total effects of predictor variables on 
TDS ranged from -0.04 to 0.50. Positive technology beliefs had the largest 
standardized total effect on TDS ((3 = 0.50), followed by technology experi¬ 
ence ((3 = 0.40). This suggests that these variables may have moderate effects 
on TDS at the classroom level, yet their cumulative effect on TDS may 
increase at the school level. Likewise, pressure appears to have a larger effect 
((3 = 0.31) on TDS at the school level that at the classroom level ((3 = 0.20). 
These results suggest that schools where on average teachers have stronger 
beliefs about technology’s benefits tend to have teachers who direct their 
students to use technology more often than do schools where teachers on 
average have weaker beliefs. 

Effects of predictor variables on TDS at the district level ranged from 0.12 
to -0.50. Integration obstacles ((3 = -0.50) had the largest effect on TDS at 
the district level, followed by technology experience ((3 = 0.40) and positive 
technology beliefs ((3 = 0.30). Note that the effect of integration obstacles 
was negative, whereas the effects of technology experience and positive 
technology beliefs were positive. Regarding standardized total effects of one 
predictor variable on another, the large effects of technology standards on 
integration obstacles ((3 = -0.50) and on pressure ((3 = 0.44) are noteworthy. 
Also noteworthy are the large effects that integration obstacles and pressure 
had on positive technology beliefs (( 3 = -0.52 and ( 3 = 0.54, respectively) and 
the large effect of a principal’s use on teachers’ technology experience ((3 = 
0.51). The multilevel model suggests that the effect of a predictor variable on 
TDS and, at times, on other predictor variables, tends to be larger at higher 
levels within the model. These results indicate that districts where teach¬ 
ers have more experience with technology and perceive fewer obstacles to 
integrations tend to have teachers who, on average, direct their students to 
use technology more often than do districts with less experienced teachers 
or teachers who report more obstacles with integration. 

Effects intensification across levels of analysis is evident when comparing 
effects between the classroom and school levels. At the classroom level, inte¬ 
gration obstacles have a -0.20 effect on TDS, whereas, at the school level, the 
variable has a -0.34 effect; integration obstacles’ effect on positive technol¬ 
ogy beliefs increases from -0.16 at the classroom level to -0.30 at the school 
level. Furthermore, pressure has a 0.33 effect on positive technology beliefs 
at the school level compared to a 0.17 effect at the classroom level. Finally, 
at the classroom level, the effect of technology standards on the integration 
obstacles is insignificant. Yet, at the school level, technology standards have a 
moderate effect on integration obstacles (-0.33). 1 This suggests that predic¬ 
tor variables’ effects may be minimal at the classroom level, but those effects 
strengthen at the school level. Comparison of school- and district-level 
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effects also suggests a magnification of effects from one hierarchical level to 
another. For instance, the negative effects of integration obstacles on TDS as 
well as those of integration obstacles on positive technology beliefs increase 
from -0.30 at the school level to -0.50 (same effect associated with both 
scales) at the district level. A practical interpretation of these results is that 
some factors may have a small impact on the extent to which an individual 
teacher uses technology in the classroom, but the overall effect of those fac¬ 
tors may increase when considering entire organizations. 

Discussion of Findings 

Despite the widespread investment in IT during the 1990s, access to tech¬ 
nology may not have translated into increased computer use (Cuban, 2001; 
Russell et al„ 2003a; Russell et al. 2003b; Wells, & Lewis, 2006).To maximize 
educational technology’s benefits for student learning, organizational lead¬ 
ers must understand which factors contribute to increased use of educational 
technology. This study aimed to identify factors that affect teacher-directed 
student use of technology in elementary classrooms by developing a multilevel 
path diagram that represents relationships between and among constructs at 
the district, school, and classroom levels to answer the following questions: (a) 
Which district-level factors contribute the most to teacher-directed student 
use of technology? (b) Which school-level factors contribute the most to 
teacher-directed student use of technology? (c) Which classroom-level factors 
contribute the most to teacher-directed student use of technology? (d) How do 
factors interact within and across organizational levels within school systems 
to contribute to teacher-directed student use of technology? 

Analyses presented here suggest that the two strongest predictors of tech¬ 
nology use at the district level appear to be (a) the extent to which district 
personnel are aware of technology standards and are held accountable to 
them and (b) principal’s discretion over technology spending. These district 
factors also appear to be associated with other important predictors of IT, 
use such as principal’s reported use of technology, teachers’ beliefs about 
the instructional benefits of technology, and perceived pressure by teach¬ 
ers to use technology. In turn, at the school level, principal’s reported use of 
technology appears to have the strongest effect on teachers’ reported use of 
technology and may affect other important predictors of classroom IT use, 
such as teachers’ beliefs about the instructional benefits of technology and 
perceived pressure by teachers to use technology. At the classroom level, re¬ 
sults suggest that the strongest predictor of reported teacher directed student 
use might be teachers’ belief about the instructional benefits of technology, 
followed by teachers’ experience with technology and teachers’ perceived 
pressure to use technology. 

1 Note that integration obstacles represent the degree to which obstacles impede the integration of technology. The negative effect of 
technology standards on integration obstacles indicates that an increase in the presence and enforcement of technology standards leads to a 
decrease in the presence of integration obstacles. 
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Researchers often cite resources and funding as barriers to IT use (An¬ 
derson & Ronnkvist, 1999; Fisher et al„ 1996; Lemke et. al, 1998). However, 
evidence from these analyses suggests that obstacles that teachers experi¬ 
enced to integrating technology into instructional practice may be just as 
detrimental. As reported here, perceived obstacles to integrating technology 
may be the biggest deterrent to directing students to use technology. Per¬ 
ceived obstacles to integrating technology, as defined by the USEIT study, 
include easy access to technology, availability of integration specialists, and 
timely access to technical support. Even when teachers have ample access 
to instructional technology because of adequate funding and access to IT 
resources, teachers may not use those resources if they do not have support. 

Another important finding reported here regards the intensification of 
effects as the analytic focus shifts from a lower to a higher organizational 
level (e.g., from the classroom level to the school or district level). Factors 
such as beliefs about instructional benefits of technology, perceived obstacles 
to integrating technology, perceived pressure by teachers to use technology, 
and accountability to technology standards may have small effects at the 
individual classroom level but may have stronger effects at the school and 
district levels. This pattern is likely due to the cumulative effect a factor may 
have on teachers who work within a school or district. For instance, not all 
teachers within a school respond identically to obstacles integrating tech¬ 
nology. Teachers who are technology innovators may reach out to external 
sources for assistance with IT integration (e.g., Internet resources, teachers 
in other districts), whereas less innovative teachers may resort solely to lo¬ 
cally provided support; these teachers may be less likely to use technology in 
instruction, thereby decreasing the district’s average use of IT. 

The incremental pattern of effects concurs with macro-level adoption 
theories. Macro-level factors have a top-down effect that may influence 
individuals within the organization in different ways but move the entire 
organization to change culture. Due to the intensification of effects at higher 
organizational levels, organizations may pay close attention to factors identi¬ 
fied in this study when considering changing the IT culture in educational 
organizations, including beliefs about instructional benefits of technology, 
perceived obstacles to integrating technology, perceived pressure reported 
by teachers to use technology, and accountability to technology standards . 

The finding that micro- and macro-level factors interact across organiza¬ 
tional levels to influence technology use in the classroom is consistent with 
Surry and Farquhar’s (1997) recommendations that successful innovation 
adoption requires targeting micro- and macro-level factors. Thus, school 
systems could consider both micro- and macro-level factors when develop¬ 
ing strategies to increase teachers’ use of technology in the classroom. To 
address micro-level factors, a school system might place greater emphasis on 
teachers’ prior technology experience when hiring teachers, promote profes¬ 
sional development opportunities that focus on the benefits rather than the 
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mechanics of using IT, and highlight IT strategies used by teachers who use 
technology innovatively and successfully. To address macro-level factors, a 
school system may consider IT use and technology standards in the develop¬ 
ment of teachers’ professional development plans, emphasize a principal’s 
prior use of technology during hiring deliberations, and allow principals to 
have discretion over their schools’ technology expenditures. Collectively, 
these macro-level actions may lead to the selection of school leaders who are 
committed to technology, have the skills to be technology advocates within 
the school, and promote a school culture that embraces technology. 

Limitations 

This study represents a step forward in the analysis of factors that contrib¬ 
ute to IT use, but it is not without limitations. SEM techniques used here 
allowed for complex data analysis, but an insufficient number of districts 
available in the USEIT data may have resulted in inflated effects at the dis¬ 
trict level. Other issues that limit generalizability are the purposive nature of 
the USEIT sample, the omission of other explanatory factors of IT use that 
were not included in USEIT surveys, and the elimination of factors from 
analyses due to measurement constraints. Additionally, analyses presented 
here focused only on one type of IT use: TDS. However, it is important 
to examine factors associated with teachers’ use of technology for other 
instructional purposes (e.g., to deliver instruction). Finally, the attribution 
of factors to specific organizational levels in analyses presented here may 
not have reflected the respective assignment of factors in school organiza¬ 
tions, as the assignment of factors in multilevel analysis is often arbitrary 
(Hox, 1998). In the future, researchers may gain a better understanding 
of factors that impact IT use by collecting data at the organizational level 
where they reside. Nonetheless, this study represents a more comprehensive 
analysis of the interaction between factors residing at various organizational 
levels of analysis than have previous studies. Perhaps more important, this 
study suggests that increasing the use of IT is not limited to a list of factors 
that individually contribute to affecting technology use in the classroom. 
Rather, results point to the importance of an entire school district culture 
committed to IT use, where district leaders set and enforce IT goals, encour¬ 
age principals to make technology-related decisions, and communicate the 
importance of using technology across the organization. 
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Appendix 



Figure A1. Initial multilevel model with school- and teacher-level predictors. 


Table A1 . Comparisons of Model Fit for Multilevel Model with Teacher- and School-Level Predictors 


Model 

X 2 

DF 

P 

X 2 /DF 

GFI 

AGFI 

NFI 

CFI 

Initial Multilevel Model 

4.13 

6.00 

0.66 

0.69 

1.00 

.99 

.99 

1.00 

Saturated model 

0.00 

63.00 



1.00 


1.00 

1.00 

Independence model 

639.46 

45.00 

0.00 

14.21 

.85 

.79 

.00 

.00 
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Teacher-Directed Student Use of Technology 



Figure A2. Multilevel model with district-level predictors, technology standards, principal’s discretion, and external funding. 


Table A2. Comparisons of Fit Indices for Multilevel Model with Technology Standards, Principal’s Discretion, and External Funding 


Models 

X 2 

DF 

P 

X 2 /DF 

GFI 

AGFI 

NFI 

CFI 

Multilevel technology standards, principal’s 
discretion, and external funding 

5.21 

12.00 

0.95 

0.43 

0.99 

0.99 

1.00 

1.00 

Multilevel model with technology standards 
and principal's discretion 

3.50 

9.00 

.94 

0.39 

1.00 

.99 

1.00 

1.00 

Saturated 

0.00 

0.00 



1.00 


1.00 

1.00 
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